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Abstract 

An analytical result and an algorithm are derived for the probabil- 
ity distribution of the one-dimensional cooperative Parrondo's games. 
We show that winning and the occurrence of the paradox depends on 
the number of players. Analytical results are compared to the results 
of the computer simulation and to the results based on the mean-field 
approach. 

1 Introduction 

Devised as a pedagogical illustration of the Brownian ratchet mechanism, 
Parrondo's games are coin nipping games with an apparently paradoxical 
property that alternating plays of two losing games produce a winning game. 
In the original setup a player has some capital, which in game A is in- 
creased by one with probability p and decreased by one with probability 
1 — p PP,[2],[II]- Game B is more complicated and the rules are that the 
probability of winning is p\ if the capital is multiple of M and if it is not, 
the probability of winning is pi- A bias is introduced into probabilities such 
that two games have a tendency to lose (capital is a decreasing function of 
the number of runs). A third game, game A + B, may be constructed as a 
random combination of games A and B, and this game is in the long run 
winning (the capital is an increasing function of the number of runs). New 
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types of Parrondo's game have been recently introduced in which the feed- 
back is introduced through spatial neighbor dependence jl], These games, 
termed cooperative Parrondo's games, rely on the state of player's neighbors 
depending whether a player has either lost or won the previous game. Each 
of N players, arranged in a circle, owns a capital Cj(i), i = 1, . . . , N, which 
evolves by combination of two games. Game A is the same as in the classi- 
cal setup, namely the probability of winning and losing is p^ and 1 — p^ 
respectively. Game B is different from its counterpart in the original setup 
since it depends on the state of the neighbors to the left and to the right of 
the player. It was shown that alternation of games A and B, which may be 
losing or fair when played individually, leads to a winning outcome. In 0], 
the evolution of probabilities in games B and C was studied using mean field 
type equations since it was assumed that the corresponding exact equations 
were too complicated to be obtained analytically. We derive here the exact 
probability evolution equations analytically and show excellent agreement 
with the results based on computer simulations. We also demonstrate that 
the existence of the paradox depends on the number of players, when a set 
of probabilities introduced in [1] is used. For arbitrary set of probabilities, 
we establish the constraints that prevent the occurrence of the paradox when 
games are played by three players. The same approach may be applied to 
arbitrary number of players however with considerable increase in complexity 
of expressions. The paper is organized as follows: Following a short presen- 
tation of essential rules of the games in section 2, we derive the probability 
transition matrix and the corresponding probability evolution equation in 
section 3. Stationary probability distribution and constraints in the form of 
inequalities for the paradox to occur are derived in section 4 and finally we 
compare analytical results with computer simulation results and the mean 
field approach of Toral jlj in section 5. We conclude with suggestions about 
possible new directions and applications of this game. 



2 Features of the Game 

Each player may be in one of two states: state ( "loser" ) or state 1 ( "win- 
ner"). The state of the whole ensemble of N players may be represented as a 
binary string s = (si, ...sn), s« G (0, 1) of length N, or equivalently as state 
s in decimal notation. We also assume periodic boundary conditions, i.e. 
Sjv+i = Si. To each state corresponds a vector, equivalent to a basis vector 
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s) in M = 2 N dimensional state space Sm 



S M = {\s) \s = 0, 1, ... 



M — 1} . 



(1) 



For example, state (Oil) is equivalent to state 3, and the corresponding 

rp 

vector is |3) = [00010000] , while state (HI) is equivalent to state 7 and 
the corresponding vector is |7) = [00000001] . Game A is the same as in 
the classical setup, while probabilities of winning in game B depend on the 
present state of left and right neighbors, denoted as a pair (sk — 1 Sfe+i), and 
with player at position k are given by: 

• p^ when (sfc_i S/t+i) = (00), in decimal notation 0, 

• p^ when (sfc_i s^+i) = (01), in decimal notation 1, 
■ p 2 when (sfc_i sj^+i) = (10), in decimal notation 2, 

• when (sfc_i s fc+1 ) = (11), in decimal notation 3. 

Winning or losing in any particular game leaves a player in state 1 ( "win- 
ner" ) or ( "loser" ) respectively, until he gets a random chance to play again. 
Capital C (t) is a function of the ensemble that is incremented by 1 or decre- 
mented by 1 if one of the players wins or loses respectively. Following a play 
by one of the players, the state of the ensemble has changed from a state s(t) 
at time t to a state s(t+ 1) at time t+1. If the probability that an ensemble 
in state s(t) (or \s(t)) ) is vr s (t), then the probability distribution p(t) at time 
t is: 



while the corresponding probability distribution evolution equation is 



It was reported in jl] that the combination of losing games A and B, leads to 
a winning result for the set of probabilities (p = = P2 — 0.16,p 3 = 0.7). 
Numerical simulations show that this set is not unique and in subsequent sec- 
tions we discuss constraints that determine this paradoxical result, however 
in further exposition we will adhere exclusively to this set. 

3 Probability Transition Matrix 

The analysis is performed via discrete time Markov chains (DTMCs) and 
we first derive the probability transition matrix for game B. Since at each 
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7T(t+l)>= T |7T(t)> 



(3) 
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moment of time only one player plays and therefore changes state, the change 
may be represented by a Hamming distance between the initial (i) and the 
final (/) state of player k is defined as: 

EN 
, Ah- fk\ ■ (4) 
k=l 

Clearly, player may either remain in the same state as before the play (dj? = 
\ik — fk\ = 0) or change his state (d^ = \ik~ fk\ — !)• We will consider each 
case separately. 
Case 1: d H = 

If the ensemble is in state s, then the k-th player is in state s k - State 
s of the ensemble also defines the neighborhood of the fc-th player, a pair 
r] k = (sfc_i, Sfc + i)which in turn determines the probability of winning. Since 
the ensemble initially in state i may switch to state f = i in one of N different 
ways as a result of one of the players switching from state i k to state fk — h, 
the probability of transition is in this case equal to the sum of probabilities 
of independent events 

1 

T fi = w(i^ f) = —2_s k=i w{i k ,f k ), (5) 

where probabilities w depend on whether state is 1 (winning) or (losing), 
and upon the probability of winning, i.e. 

w^kJk) = l 1 S '* = ? (6) 

{ PvA fk = i 

Case 1: d H = 1 

In this case k-th player switches from state i k to state f k , (ik ^ fk ), 
with probability 

Tfi = ^*>(ik,fk). (7) 

3.1 Probability transition matrix for = 3 

As an example we consider an ensemble of three players (N = 3) playing 
game B, and let us assume that the ensemble is in state s = (011)= 3. 
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Possible transitions and the corresponding probabilities are: 
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[(1 -p 3 ) +Pi + P2] 

;i-p 2 ) 



(8) 



Explicitly, probability X13 represents the probability that second player, in 
state S2 = 1, switches to state / 2 = 0. This probability is equal to the 
product of the probability that it is this player's turn to play i.e. 1/3, and 
the probability that this player loses, i.e. switches to state / 2 = 0. The 
neighborhood of the second player, (si S3) = (0 1) = 1, determines the 
choice p\. Similarly, transition T 33 may take place when either one of the 
three players switches to a same state, hence the corresponding probability 
is the sum shown in the first expression of |H1 Finally, using 
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Corresponding matrix for game A may be easily obtained by performing the 
following replacement: plf — > for each p G {0, 1, 2, 3}. 

Furthermore, we introduce a vector of the capital \C) whose components 
represent the capital generated by each ensemble state. Elements C s of this 
vector represent normalized capital generated by that specific state (equal 
to the sum of all winning and losing individual states in a given ensemble 
state). Naturally, player state generates capital —1, while state 1 generates 
capital +1. Explicitly, 



1 



-1 



so that C s E [-1, 1] . 



(10) 



5 



In other words, elements of \C) are average values of the capital generated 
by each ensemble state separately. For example for N = 3, the vector of the 
capital is: 

\C) = (1/3) [-3 - 1 - 1 1 - 1 1 1 3] T . (11) 

In the above expression the third vector element corresponding to the state 
(010) =2 is equal to C 2 = (l/3)((— 1) + (+1) + (-1)) = (l/3)(-l). This also 
implies that the ensemble switching from a state s(t) to a state s(t + 1) = 
(010) = |2), under the assumption that such a transition is possible, gener- 
ates average capital (C(t + 1)) = (C | 2) = —1/3. Furthermore, an ensemble 
remaining in state |2) throughout its temporal evolution would in the aver- 
age generate capital (—1/3) in each turn of the game. Hence the capital 
generated by the ensemble is 

(C) = (C | tt) , (12) 

where (C) denotes the average value of the generated capital. In order to 
evaluate the probability for one of the games, either B or A+B to be winning, 
it should be noted that: 



Pwin Plose 1 (13) 

Pwin Plose (C) 

where P win and Pi ose are probabilities of winning and losing in a certain game, 
so that 

P wm = (l/2)(1 + (C». (14) 

This expression implies that condition < 1/2 , i.e. that game B is 

losing, is equivalent to the condition (C^) < 0, and that > 1/2, i.e. 

that game A + B is winning, is equivalent to (C^ A+B ^) > 0. 



4 Analysis of the Games 
4.1 Equilibrium distribution 

The equilibrium (stationary) state occurs when the probability distribution 
remains invariant under the action of T, that is, 1 7r (t -h 1)) = T \ir(t)) = 
\ir). In order to evaluate the probability distribution in this case, we need 
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to solve (1 — T)ir = 0. For game A, for which there is a probability p for 
a player to win (alternatively (1 — p) to lose), the stationary distribution is 
easily obtained by setting p = p\ = p 2 = p% = p and reads 

tt< a > = [(1 -pf,{l-pfp,{l-pfp, (1 -p)p\ (15) 
(l-pYp,(l-p)p\(l-p)p\pY (16) 

The probabilities in the above expression may also be readily obtained by 
associating to each ensemble state, from to 7 in decimal notation, corre- 
sponding probabilities for each player. For game B the stationary distribu- 
tion is 

n (B) = r (l ~Po)[2 - (gi +p 2 )](l -p 3 )a [2 - (gi +p 2 )](l -Pz)a ^ 

PoP3(pi+P2) pm(pi + P2) 

[2 - ( Pl + p 2 )](l - p 3 )a q(l -p 3 ) [2- (p 1 +p 2 )](l -p 3 )q 

P0P3(Pl+P2) P3 P0P3(Pl+P2) 

a(l-p 3 ) a(l-p 3 ) T 
, ,a , 



P3 P3 



where 



a [(i-Pu)+3p 3 ](i-P3)[2-(pi+P2)](i-P3) _ 3(i- P3 ) , 1 " ( 18 ) 

P0P3(P1+P2) P3 

Substituting the set of probabilities given in 0], Po = l,Pi = P2 = 0.16, p 3 = 
0.7, in the expression for the stationary distribution for game B, we get 

tt (b) = [0, .24901, .24901, .04743, .24901, .04743, .04743, .11067] T (19) 

For the randomized game A + B, probabilities Pi(i = 0, 1, 2, 3) for game B, 
are replaced with the corresponding probabilities qi(i = 0, 1, 2, 3) given by: 

g, = 7P+(l-7) Pi , (z = 0,l,2,3). (20) 

and where parameter 7 represents the relative probability of playing game A, 
where we have assumed the value of one half. The corresponding stationary 
distribution for the randomized game, with p — 1/2, is 

r (A+B) = [ 06006, .18019, .18019, .08875, .18019, .08875, .08875, .13312] T . 

(21) 

Expressions EH and |^ although not leading to the paradoxical result illus- 
trate the probabilities associated with each state in games B and A + B 
respectively. 
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4.2 Constraints of the games 

The probability of winning using the stationary distribution is given by 

EA/-1 
vr s p s , (22) 
s=0 

where p s is the winning probability in state tt s . Components p s of vector \p) 
may be defined using El or alternatively as, 

- EL f w 

so that the probability that the ensemble generates a winning outcome when 
switching to a state s — (si, . . . , Sjv), is determined by the fraction of winning 
players in the ensemble. The summation in the above expression should 
actually be over the indices corresponding to the winning players, however 
use of all indices is justified since states do not contribute to the summation. 
Since for the paradox to occur we must have P^ < 1/2 and P^^ > 1/2 
simultaneously, a simple calculation yields the following inequality 

(pi + P2)[l + \(po- Ps)] > 2[1 - ^(p + 3p 3 )]. (24) 

We may also assume that probabilities pi and p 2 should be equal, so that 
finally we get 

1 - Kgo + 3p 3 ) 
1 > pi > \- — . 25) 

If po = 1) an d P3 = 0.7 are inserted in the above expression, the inequality 
is not satisfied and there is no paradox, justifying the conclusion based on 
numerical simulations that there is no paradox for games played by three 
players when probabilities are those given by Toral in However, for the 
same number of players and a set of probabilities, e.g. po = 0.09, pi = P2 = 
0.52 and = 0.89, the paradox exists. 



5 Results 

All results in this section are based on probability values given in 0j, namely 
(p = l,pi = P2 = 0.16, p 3 = 0.7, = 0.499). First, in Fig. 1 we present 
analytical results that show how paradox depends on the number of players 
when iV < 12. Namely, in this range of iV values paradox exists if iV is 
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different from 3,4,7 or 8. Numerical simulations, shown in Fig. 2, indicate 
that the paradox occurs if the number of players is greater than 8 and no 
exceptions are noticed up to 1000 players. The mean-field equation for the 
evolution of the common probability in game B 

P^ B \t+l) = (1- p( B )(t)) 2 Po + P (B) (t)(l- p( B \t))( Pl +p 2 )+p( B \t) 2 p 3 , 

(26) 

in which each term reflects one of the four possibilities given in the definition 
of the game, introduced in naturally yields results irrespective of the 
number of players. 

As a comparative illustration of results based on all three approaches, we 
show in Fig. 3 the average capital per turn as a function of the probability p% 
of game B played by five players. Analytical and numerical simulation results 
show perfect agreement while results based of the mean field equation show 
very good agreement only in the (approximate) range 0.16 < p% < 0.83. 
For values of p% below 0.16 the capital based on mean field calculations 
shows that there is no stable solution of the iterative mean field equation. 
Namely, mean field equation in this area switches to one of the two solutions 
at each successive step of calculations while it deviates considerably from 
the analytical and numerical results for p 3 > 0.83. We emphasize that due 
to the very nature of the mean-field approach this large discrepancy for low 
and high values of p% remains for arbitrary number of players. However the 
dependence of the average capital as function of other probabilities, po,p± 
and P2 is considerably better, as presented in Fig. 4. for the case of po . 

6 Conclusion 

An algorithm for obtaining analytical expressions for the evolution of the 
probabili-ties and constraints of the games is illustrated in the case of three 
players. These results may prove to be of important use in applications 
to social and economic models where exact results may be indispensable. 
Usefulness of these games to social and economic models, and possibly in 
biological applications, may be expanded by analyzing games played by all 
players simultaneously that we introduce in the follow up paper . Also, the 
approach presented here may be useful in analysis of games based on spin 
models. Moreover, we show that the mean-field approach, although useful 
when large number of players is involved, may be quite inaccurate in certain 
range of probabilities that define spatial neighbor dependence. 
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Fig. 1. Averaged capital per turn as a function of the number of players 
obtained using analytical expressions for the evolution of the probability dis- 
tribution.. For 1 < N < 6 it is easy to distinguish winning and losing games, 
however for N > 7game B is barely losing and in order to accentuate that con- 
clusion we use magnified scale for the capital. The spline fit is performed solely 
for visualization purposes. 

Fig. 2. Numerically obtained averaged capital per turn (realization of the 
games) as a function of the number of players (N). Capital is averaged over 
1000 runs of the game, and over 1007V turns in each run after the system entered 
the stationary state. 

Fig. 3. Mean-field approach results compared to the analytical and numer- 
ical results for the average capital per turn as a function of p\ ' for N = 5, 
(po = l,pi =P2 = 0.16). Analytical and numerical results show excellent agree- 
ment while mean-field results show considerable divergence for small and large 
values of p^ . 

Fig. 4 Average capital per turn of the game as a function of p . Due to 
the large number of players only results based on the computer simulations are 
compared to the mean-field results. The other probabilities Pi = P2 = 0.16, 
P3 = 0.7 are kept fixed. The graph shows very good agreement. 
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